1. Field of the Invention
This invention relates to digital photography, and more particularly to a frequency compensation technique for a digital video camera.
2. Description of Related Art
Digital photography is one of the most exciting technologies to have emerged during the twentieth century. With the appropriate hardware and software (and a little knowledge), anyone can put the principles of digital photography to work. Digital cameras, for example, are on the cutting edge of digital photography. Recent product introductions, technological advancements, and price cuts, along with the emergence of email and the World Wide Web, have helped make the digital cameras one of the hottest new category of consumer electronics products.
Digital cameras, however, do not work in the same way as traditional film cameras do. In fact, they are more closely related to computer scanners, copiers, or fax machines. Most digital cameras use an image sensor or photosensitive device, such as a charged-coupled device (CCD) or a complementary metal-oxide semiconductor (CMOS) sensor to sense an image. An array of these image sensors are arranged in the focal plane of the camera such that each sensor produces an electrical signal proportional to the light intensity at its location.
The image thus produced has a resolution determined by the number of sensors in the array. A modern digital camera may have a million or more of these sensors. The resulting image will be digital, having picture elements (pixels) corresponding to the number of sensors in the array. Because of the correlation, the sensor elements themselves are often referred to as pixels as well.
Sensor arrays are known in many forms. One common one is a two dimensional form addressable by row and column. Once a row of elements has been addressed, the analog signals from each of the sensors in the row are coupled to the respective columns in the array. An analog-to-digital converter (ADC) may then be used to convert the analog signals on the columns to digital signals so as to provide only digital signals at the output of the array, which is typically formed on an integrated circuit.
Because of a number of problems such as degradation of signal and slow read out times in prior art sensor arrays, a xe2x80x9cdigital sensor pixelxe2x80x9d has been developed as described in, e.g., U.S. Pat. No. 5,461,425, which is hereby incorporated by reference. FIG. 1 illustrates an array 12 of digital sensor pixels 14 on an integrated circuit 10. Each digital sensor pixel 14 in the array 12 includes a photodiode and a dedicated ADC such that the pixel 14 outputs a digital rather than an analog signal as in prior art sensor arrays. In contrast, prior art sensor arrays did not have a dedicated ADC for each individual sensor in the array. Digital filters 16 on integrated circuit 10 are connected to receive the digital output streams from each digital pixel sensor 14 and convert each stream to, e.g., an eight-bit number representative of one of 256 levels of light intensity detected by the respective digital pixel sensor 14. Within the digital pixel sensor 14, the analog signal from the photodiode is converted into a serial bit stream from its dedicated ADC clocked using a common clock driver 18. The digital filters 16 process the bit stream from each digital pixel sensor 14 to generate an eight-bit value per pixel element 14. These eight-bit values may then be output from the chip 10, using a suitable multiplexer or shift register, and temporarily stored in a bit-mapped memory 24.
Because a digital signal is produced directly by the pixel 14, several advantages over the prior art become apparent. For example, dynamic range is a critical figure of merit for image sensors used in digital cameras. The dynamic range of an image sensor is often not wide enough to capture scenes with both highlights and dark shadows. This is especially the case for CMOS sensors that, in general, have lower dynamic range than CCDs.
To address the need for increased dynamic range, U.S. Ser. Nos. 09/567,786 and 09/567,638, both filed May 9, 2000 and incorporated by reference herein, disclose an architecture for the digital pixel sensor in which the dynamic range of the sensor is increased by taking multiple samples of a subject during a single imaging frame, where each sample is taken over an interval of a different duration (integration time) than the other samples. As will be described in greater detail herein, such a multiple sampling architecture avoids limitations in dynamic range as experienced by prior art CMOS sensor arrays. However, despite this advantage, this multiple sampling scheme will introduce certain distortions when implemented in a video camera. These distortions arise between the human-perceived image and that recorded by the video camera. For example, consider the image recorded by a single pixel in a video camera implementing a multiple sampling scheme where the image light intensity varies with time. Because the image light intensity is time-varying, the multiple sampling scheme may alter the effective integration time from frame-to-frame for this pixel. For example, at a first frame the integration time may be xe2x80x9cTxe2x80x9d seconds long whereas at a second frame the integration time may be 8T seconds in length. In contrast, the human-perceived image may be modeled as having a fixed integration time for these same samples, resulting in distortion between the human-perceived and digitally-recorded images. This distortion will be evident in other digital video systems which do not practice a multiple sampling method but do use different exposure times for a given pixel from frame-to-frame.
Accordingly, there is a need in the art for a video digital camera that benefits from the increased dynamic range afforded by a multiple sampling scheme without suffering distortion with respect to a human-perceived image.
In accordance with one aspect of the invention, a method of frequency compensation is presented for a video system using an exposure time selected from a set of exposure times for a given pixel. The video system selects the exposure time for the given pixel such that the exposure time varies from video frame to video frame. The resulting image signals from the given pixel will thus be formed with varying exposure times. This method calculates a complete set of image signals for any exposure time selected from the set of exposure times for the given pixel.
In accordance with another aspect of the invention, a frequency-compensated video image system using a time-indexed-multiple-sampling technique is presented. The video image system includes a digital sensor pixel that, in response to an image signal, selects from a plurality of exposure times to form a digital image signal in a given video frame. A memory couples to the digital pixel sensor for storing the digital image signals corresponding to a plurality of video frames, wherein the memory also stores the corresponding exposure time selected for each digital image signal in a given video frame. A processor calculates, from the stored plurality of digital image signals and their corresponding exposure times over the plurality of video frames, a compensated digital image signal for each video frame in the plurality of video frames, wherein the compensated digital image signal uses a constant exposure time from video frame to video frame.
The invention will be more fully understood upon consideration of the detailed description below, taken together with the accompanying drawings.